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^ ■ Abstract 

I We construct a generic, simple, and efficient scheduling policy for stochastic processing 

networks, and provide a general framework to establish its stability. Our policy is randomized 
and prioritized: with high probability it prioritizes jobs which have been least routed through 
the network. We show that the network is globally stable under this policy if there exists an 
appropriate quadratic 'local' Lyapunov function that provides a negative drift with respect 
. to nominal loads at servers. Applying this generic framework, we obtain stability results for 

our policy in many important examples of stochastic processing networks: open multiclass 
queueing networks, parallel server networks, networks of input-queued switches, and a variety 
of wireless network models with interference constraints. Our main novelty is the construction 
of an appropriate 'global' Lyapunov function from quadratic 'local' Lyapunov functions, 
which we believe to be of broader interest. 
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The past few decades have witnessed a surge in interest on the design and analysis of scheduling 
policies for stochastic networks, e.g., [I3l [Ml [lEl EHl US [5]. One of the key insights from this 
CN ■ body of work is that natural scheduling policies can lead to instability even when each server 

is nominally underloaded \15\ \T7\ I20j . (There are several notions of stability for stochastic 
networks, but they intuitively entail that, in some sense, the number of customers in the system 
^ ■ does not grow without bounds.) This insight stimulated a search for tools that can characterize 

the stability regions of scheduling policies, i.e., the exact conditions on the arrival and service 
rates under which a network is stabilized by a policy. 

It is the objective of this paper to study a question of a different kind: is it possible to 
construct a generic, simple, and efficient scheduling policy for stochastic processing networks, 
which leads to a (globally) stable network if all servers are (locally) nominally underloaded in 
some sense? To our knowledge, we are the first to answer this question within the setting of 
stochastic processing networks, which constitute a large class of stochastic networks capable of 
modeling a variety of networked systems for communication, manufacturing, and service systems 
(e.g., |12j). To investigate this question, the key is to determine if and how jobs from different 
parts of the network should be treated differently when they share the same buffer. 
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Various existing scheduling policies are 'throughput optimal' in the sense that they achieve 
the largest possible stability region, but these policies suffer from significant drawbacks. They 
typically obtain the desired stability by framing the contention resolution between buffers as an 
appropriate global optimization problem. This optimization problem requires central coordina- 
tion between network entities, and it is computationally hard to solve if the network is large. 
The resulting policies, such as the max- weight policy [I8l[23] and the back-pressure policy [24115]. 
are not scalable and cannot cope efficiently with large networks. As a result, these throughput 
optimal policies do not provide a satisfactory answer to the aforementioned question. 

The computational challenges surrounding existing throughput optimal policies motivate the 
search for easily implementable scheduling policies with provable performance guarantees but 
not necessarily with the throughput optimality property. This has led to the analysis of simple 
greedy scheduling policies for a variety of special classes of stochastic processing networks: open 
multiclass queueing networks [3], input-queued switches [6], and wireless network models [16] . 
These policies play an important role in this paper, since our most critical assumption roughly 
requires any local network component to be nominally underloaded under any 'maximal' greedy 
scheduling policy. 

Our main contribution is a randomized scheduling policy for stochastic processing networks 
which only requires coordination within local components (e.g., service stations), and which is 
computationally attractive since it is a kind of priority policy. With high probability, our policy 
prioritizes jobs which have been least routed, and we therefore call our policy the e-Least Routed 
First Served (e-LRFS) policy. Here e > is a small number which helps to make the meaning 
of "high probability" precise. 

Our main technical tool is a novel framework to construct a 'global' Lyapunov function for 
a stochastic processing network through appropriate 'local' Lyapunov functions. If the local 
Lyapunov functions yield stability of the corresponding 'local' network components, then the 
global Lyapunov function allows us to conclude that the whole network is stable. A critical 
feature of our framework is that the Lyapunov functions we work with are quadratic. Through 
examples, we show that quadratic local Lyapunov functions can readily be found for wide classes 
of networks. We refer to |8l [TJj for other uses of quadratic Lyapunov functions. 

Our approach to construct an appropriate 'global' Lyapunov function using 'local' quadratic 
Lyapunov functions contrasts with the popular fluid model methodology for establishing stability 
of stochastic networks [Sj [U [20]. The fluid model framework essentially reduces the question 
of stochastic stability to a question of a related deterministic (fluid) system. In the case of 
reentrant lines, our policy reduces to the First Buffer First Served (FBFS) policy, which has 
been proven to be stable via 'inductive' fluid arguments [7]. A similar fluid induction argument 
can be expected to work for general multiclass networks (modulo some technical arguments), but 
a fluid induction argument cannot be expected to work in general. A disadvantage compared to 
fluid models is that we have to keep track of detailed system behavior such as remaining service 
times, but therein also lies the power of our approach. The generality of our framework presents 
challenges to the use of fluid methods, and instead we work directly with a global Lyapunov 
function. The connection with fluid techniques is discussed in more detail in Section [4.31 

Although we believe that our methodology could be of much wider use, we have chosen to work 
out two special classes of stochastic processing networks in order to describe the implications 
of our techniques in relatively simple yet powerful settings: parallel server networks (including 
multiclass queueing networks) and communication network models (including networks of input- 
queued switches and wireless networks). 
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Our work is related to a paper by Bramson [2], who shows that open multiclass queueing 
networks can be stabilized by the Earliest Due Date First Served (EDDFS) policy under the 
'local' condition that every processing unit is nominally underloaded. However, it is not known 
whether the EDDFS policy achieves similar results beyond the open multiclass queueing network. 
In fact, it is a priori unclear how to formulate a 'local' condition for general stochastic processing 
networks; our notion of 'local' Lyapunov function plays this role in the present paper. 

This paper is organized as follows. Section [2] introduces the class of stochastic processing 
networks we study in this paper. Section [3] formally introduces the e-LRFS policy and presents 
our main results. Section U] describes our main idea using a simple network, the Rybko-Stolyar 
network [20]. We specialize our result to parallel server and communication network models in 
Section [5l All proofs are given in Section [6l 

2 A Class of Stochastic Processing Networks 

A stochastic processing network (SPN) consists of a set J = {1, . . . , /} of buffers, a set 3 = 
{1, . . . , J} of activities and a set % = {1, . . . , K} of processors. Each buffer has infinite capacity 
and holds jobs that await service. The SPNs we study in this paper have the feature that 
activity j S can only process jobs from a single buffer ij € 3, and that j requires simultaneous 
possession of a set JCj C 3C of processors. Let 3i be the set of activities capable of processing 
buffer i, i.e., 

3i = {j e3- ij = i}- 

We say that two buffers i and i are activity-interchangeable if {Xj : j G 3i} = {Xj : j G 3e}- We 
also say that buffers i and i are processor-independent if Ujgj ^j£3e ^'■^ disjoint. 

Network state. We let Qi{t) G Z+ be the queue length of buffer i at time t, i.e., the number 
of jobs waiting in buffer i excluding those being processed. We write Vi{t) G M+ for the sum of 
the remaining service requirements over all jobs in buffer i which are currently being processed 
at time t. We use Uj (t) to denote the activity level of activity j at time t, where we assume that 
a{t) = [uj{t)\ G {0, 1}"', meaning that each activity is either fully employed or not employed at 
all. Thus, we say that the network is non-processor-splitting. 

Routing. After departing from a buffer i, a job joins buffer £ G J with probability Pn and 
departs from the network with probability 1 — Pm (independently of everything else) . We 
write P for the / x / matrix of routing probabilities. 

Resource allocation. Each activity j decreases the remaining service requirement of the job 
it is processing at rate /3j > if (Tj{t) = 1. It is not allowed for an activity to be interrupted 
before it finishes the service requirement of the job it is working on, i.e., the network is non- 
preemptive. We do not allow for multiple activities to work on the same job simultaneously. 
Furthermore, we assume that each processor k has unit capacity, i.e.. 

We note that this unit capacity assumption is not restrictive since in our non-processor-splitting 
network of a{t) G {0, 1}^, a single processor k with capacity G N can be replaced by copies 
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with unit capacity, and identical activity structure inherited from the original processor. We 
further define the service rate vector s{u) = [si{u)] € M;^ for a (scheduling) vector u = [uj] G 
{0, 1}"' through 

j€3i 



sdu 



External arrivals. There are external arrivals to at least one buffer. For < s < t, let Ai{s, t) 
be the number of external jobs arriving at buffer i during the time interval [s,t). We assume 
that E [Ai{0,tf] < oo for all t < oo (i.e., bounded second moment) and 



limsup^; Ai{t,t + 1) {Ai{0,s),...,Aj{0,s) :0 < s <t} 



t—>-oc 



for constant ai € (0, oo). We note that we allow for dependencies in the random processes Ai(-) 
and A£{-) for two buffers i and i. Even though it is possible for the to exceed the external 
arrival rates, it is convenient to interpret Oi as the external arrival rate at buffer i. Similarly 
abusing terminology, we let A = [Aj] be the effective arrival rate vector, i.e.. 



a. 



X .- ^I + P + P^ + 

where cx = [ai]. We also say that all routes are bounded (in length) if inf |c? G N : P'^ = O} < oo. 



Service requirements. Once a job in buffer i is selected for processing by activity j G (Jj, it 
requires service for a random amount of time. We assume that all service times are independent, 
and that they are independent of the routing and external arrival processes. We also suppose 
that the service time distribution only depends on the buffer from which the job is processed. 
Writing Fj for a generic processing time at buffer i, we assume that 

E[Ti] = rui and E [Ff] < oo, 

for constants € (0, oo). Let p = [pi] denote the nominal load, i.e., pi = Xirrn. We write Wi{t) 
for the (expected) immediate workload in buffer i, which we define to be 

W^{t) = miQi{t) + Vi{t). (1) 

This is the expected amount of work in the i-th buffer given Qi{t) and Vi{t). We note that our 
notion of immediate workload is the conditional expectation of a more common definition, and 
that immediate workload is defined for each buffer (as opposed to resource). We say that the 
network is synchronized if for all s, t > 0, i G J and j G J, 

Ai[s^t) = Ai{s^ \ ^-iid Fj = mj = Pj = 1 with probability 1. 

That is, in a synchronized network, arrivals and service completions only occur at integer time 
epochs. 



Maximal scheduling policies. We say that activity j is maximal in a (scheduling) vector 
u = [uj\ G {0, 1}"^ if there exists a processor k G %j such that 
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i.e., either activity j uses each of the processors in 3Cj under schedule u or it cannot be employed 
without violating the unit capacity constraint for some processor in %j. Given a non-negative 
vector w = [wi] S M;?^, activity j is called maximal in u with respect to w if Wi- = or j is 
maximal in u. An activity is non-maximal if it is not maximal. If all activities are maximal in 
u with respect to w, we simply say that u is maximal with respect to w. We write M(it)) for 
the set of maximal scheduling vectors with respect to w, i.e., 

M{w) := {u G {0,1}-^ : u is maximal with respect to lu} . 

Finally, we say that a scheduling policy is maximal if, under the policy, cr{t) G M(Q(i)) for all 
t > 0. 



3 Main Result 

This section describes the scheduling policies which play a central role in this paper, and presents 
our main stability result. 



Scheduling policies. Our policies require that each job maintains a 'counter' for the number 
of times it has been routed so far, where we follow the convention that counters start from 1. 
The counter of a job is increased even when a job is routed to a buffer it has previously visited, 
so the counter of a job could differ from the number of different stations it has visited. We also 
consider a partition {3^^^ : h £ = {1, . . . , H}} of buffers into components, such that 

3= U JW, 

where any two buffers from different components are processor-independent. One possible choice 
for the partition is J = 3^^\ but, as becomes apparent from the description of our policies below, 
a finer partition makes our policies more 'distributed'. It is important to note that we allow 
for routing between components. We assume that each component 3^^^^ for h "K maintains a 
'timer' 7^^\t) G [0, 1] at time t which decreases at unit rate if 7^^\t) > 0. 

To describe our maximal scheduling policies, we need the following notation. We write 
Q^'^Ht) = [Qiit)Ijwii)] for the queue length information in component 3^^\ where Is G {0, 1} 
is the indicator function of the set S, i.e.. 



Is{i} 



1 iiieS 
otherwise 



The following policy plays a key role throughout this paper. 

Definition 3.1 (Least Routed First Served (LRFS) policy). For each h £ 3i, whenever a 
new arrival and/or service completion occurs at time t in component 3^^\ execute the following 
algorithm immediately after all arrivals and service completions have occurred: 

1. Find the set S of non-maximal activities in a{t) with respect to Q^'^^t). 

2. Find a job with the smallest counter among those in buffers {ij : j G S}, where ties are 
broken arbitrarily. 
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3. Choose an arbitrary activity j & T, to process the job identified in step 2, i.e., setaj{t) = 1. 

4- Repeat steps 1-3 until a{t) is maximal with respect to Q^^\t). 

We next introduce the maximal scheduling policy which is of primary interest in this paper. It 
uses a small parameter e > in order to deal with unbounded route lengths. 

Definition 3.2 (e-Least Routed First Served (e-LRFS) policy). For each h ^ "K, when- 
ever a new arrival and/or service completion occurs at time t in component 3^^'^, execute the 
following algorithm immediately after all arrivals and service completions have occurred: 

1. Find a buffer i € J^^^ containing a job with the largest counter among those in buffers J^'^). 

2. Find the set of non-maximal activities S C 3i in cr{t). 

3. IfT.^$ andl^'^Xt) = 0, 

3-1. With probability e, choose an arbitrary activity j to process the job identified in 
step 1, i.e., set cTj{t) = 1. With probability 1 — £, do nothing. 

3-2. Setl^^Xt) = 1. 

4- Execute the LRFS policy for component 

We remark that the e-LRFS policy is identical to the LRFS policy when either e = or T^'^-* (t) > 
0, i.e., execution of the first three steps is not necessary in these cases. Since we assume that 
each timer decreases at unit rate, step 3-1 can be executed at most once per component in any 
time interval of unit length. 

The network process. Write Qi^dt) for the queue length of jobs with counter c in buffer i 
at time t. Let V/^t) be the remaining service requirement of the job with counter c in buffer 

i at time t if it is processed by activity j, and set V/^t) = if j is not processing a job with 
counter c. The network state is described by 

Xit) = [Qi,,(t),y4(t),TW(t) : ied,j e8,ceN,heJi^ g l^x := (^i x ]R^^^)°° x [O, l]^. 

Note that X{t) does not encode information on the external arrival processes. In particular, 
{X{t)} is non-Markovian in general. We impose the convention that {X{t)} has right-continuous 
sample paths. We define a norm on Qx through 

1^(01 = E ^^'.^(*)+ E 

We assume that X{0) € where O,'^ = {X € 0,x : |^| < cjo}. In synchronized networks, 
one has V^^t) G {0, 1} for all t G Z+ under the e-LRFS policy. We refer to the process 
{X{t) : t G M+} operating under the e-LRFS policy as the e-LRFS process. If e = 0, then we 
simply refer to this process as the LRFS process. 
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(2) 



for any given initial state X{0) G 0^. 

We establish the queue-length-stability by constructing appropriate Lyapunov functions. Un- 
der some additional assumptions on the arrival processes and service time distributions, these 
Lyapunov functions can also be used to establish positive recurrence of the e-LRFS process, 
cf. Condition (A3) in [4]. It is outside of the main scope of the current paper to work out the 
details. We also remark that our proof can find an explicit finite constant for the right-hand side 
of ([2]), but this requires tedious bookkeeping and we therefore do not carry out this analysis. 

To show the desired stability, we need the following notion of a 'local' Lyapunov function. 

Definition 3.4. We say that ^locai '■ ~^ ^+ is a local Lyapunov function with slack parameter 
e > if there exist constants C > 0, > such that for every pair {w^u) € ]R:'^x{0, 1}"' satisfying 



The reason for this nomenclature is that in a 'local' network, i.e., a network without routing 
{P = 0), the above inequality for £iocai provides the desired negative drift condition in the 
Foster-Lyapunov criteria [10], which implies network stability. Here Wi, pi, and Si{u) can be 
interpreted as the immediate workload, the external workload arrival rate and the workload 
processing rate at buffer i, respectively. We refer to Sections [4] and [5] for examples of local 
Lyapunov functions. 

Now we are ready to state the main theorem of this paper, which establishes 'global' stability 
using a 'local' quadratic Lyapunov function. 

Theorem 3.5. Suppose that there exists a symmetric matrix Z G M^^^ such that £iocai(3^) = 
x'^Zx is a local Lyapunov function with slack e > 0. Then the e-LRFS process is queue-length- 
stable if one of the following conditions CI and C2 is satisfied: 

CI. The network is synchronized and Zn ^ only if buffers i and I are in the same component. 

C2. Zii 7^ only if buffers i and I are activity-interchangeable and every buffer i has an 
associated activity j € di with \%j\ = 1. 

Furthermore, if all routes are bounded, then the LRFS process is queue-length-stable if one of 
the conditions CI or C2' is satisfied, where 

C2' . Zii 7^ only if buffers i and i are activity-interchangeable. 

Theorem 13.51 implies that if any maximal policy is 'quadratic' stable in a stochastic processing 
network without routing under the external load p, then e-LRFS is stable in a stochastic pro- 
cessing network with routing under nominal load < p for some small e > 0. Section H] describes 
the main idea of the proof, and a full proof is presented in Section [6l 

We remark that the requirement of activity-interchangeable buffers in Condition C2 can be 
relaxed slightly. Our proof of Theorem 13.51 also works when the following relaxed condition C3 
replaces C2. 



local If + P + e"^ - s(u) < Lioc!a{w) - ri\\w\\i + Q, 



(3) 



where m 



[mi] G M-^. 
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C3. Zii ^ only if for every j G 2i, there exists j' G 'Si such that %ji C DCj, and vice versa 
(i.e., for every j e 0^, there exists j' € ^Jj such that %ji C Also, every buffer i has an 
associated activity j G 2% with = 1. 

A corresponding condition C3' can also replace C2' , where C3' does not require the second part 
of C3. 

4 Proof Ideas for Theorem 13.51 

In this section, we describe the main idea in the proof of Theorem 13.51 We first present it 
in a very special network, the Rybko-Stolyar network [JOj, which allows us to summarize the 
main idea of the proof at a high level. We subsequently describe the challenges that have to be 
overcome to establish our result in the general case, and discuss the feasibility of an approach 
based on fluid models. 

4.1 Rybko-Stolyar Network 

This network consists of four activities associated to four different buffers and two processors. 
The first processor is required for activities 1 and 4, and the second for activities 2 and 3. Each 
activity decreases the remaining service requirement of the job it is currently processing at unit 
rate (i.e., /3j = 1). Customers (or jobs) arrive at the first and third buffers, and traverse the 
buffers deterministically in the order 1 — ?• 2 or 3 — ?> 4. The service time is deterministic and 
equal to rrii for buffer i, and the external arrival processes (at the first and third buffers) are 
independent Poisson processes with rate 1. The network is given in Figured] and a necessary 
condition for stability is 

pi = mi + rui < 1 and p2 = m2 + < 1. (4) 



rate 1 



rate 1 



Figure 1: Illustration of the Rybko-Stolyar Network. In this diagram, there are four jobs in the 
system. Jobs are currently being processed and waiting for service are colored black and white, 
respectively. 



The network state can be described by X{t) = {Qi{t), Vi{t) : i = 1, 2, 3, 4} G Qx ■= Z+ x R\, 
where Qi{t) is the queue length (i.e., the number of jobs waiting) of buffer i at time t and 
Vi(t) is the remaining service time of the job currently being processed in buffer i at time t. 
Hence Vi{t) = if no job is being processed in buffer i. We define a norm through |X(t)| = 
+ Qi{t))- III this network, it is known [2^ that if processor 1 prioritizes buffer 4 and 
processor 2 prioritizes buffer 2, the network can be unstable even when the necessary condition 
([4]) holds. On the other hand, LRFS prioritizes buffer 1 and 3, so it reduces to the First Buffer 
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First Served policy. This is known to be stable under the necessary stability condition (jH). We 
next derive this stability result using our main idea. 

Assuming LRFS and we construct an appropriate Lyapunov function -Cgiobai : ^+ 
satisfying 



E 



■^global (-'^(i + 1)) — ■C'glohal{X{t)) 



< -^\X{t)\ + Q, almost surely, (5) 



where 7, C > are some constants. By first taking expectations with respect to the distribution 
of X{t) and then integrating over t on both sides of the above inequality, we conclude that 



t^oo 7 V t I 



e 

7 



< 00. 



(6) 



We now proceed toward constructing the 'global' Lyapunov function £giobai satisfying ([5]) 
based on a 'local' quadratic Lyapunov function £iocai- To this end, we first discuss how to 
construct the 'local' quadratic Lyapunov function. Consider a single-processor system with two 
buffers a and 6, deterministic service times given by and mf,, and independent Poisson arrival 
processes with rate 1 at each buffer (i.e., the total rate is 2). Hence, a necessary condition for 
stability is 

ma + mh < 1. 

Under a maximal (i.e., work-conserving) scheduling policy, the workload W{t) = VFj(t) at 
time t satisfies 

W{t + 1) = W{t)+^miAi{t,t + l)- l''^\+,^^^du, 

i=a,b * 

where Wi{t) is defined as the immediate workload at buffer i as in ([T]), Ai{s,t) is the number of 
jobs arriving at buffer i during the time interval [s,t) so that E[Ai{s,t)] = t — s and we define 

^+ _ h ifa;>0 
^ 1 otherwise 



Hence, we have 



E 



W{t + 1) -W{t) W{t) 



< ma + mb 



(7) 



where we use that l^j-^) > ^or u G [t, since we assume deterministic service times. 
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Using this, it follows that for some finite constant C, 



E 



W{t + lf -W{t)' 



W{t) 



E 



2W{t){W{t + l)-W{t)) 



W{t) 



+E 



< 2W{t)E 



{W{t + 1)-W{t)f 



Wit) 



W{t + 1) -W{t) 



Wit) 



+ e 



< 2Wit) [ma + ITLb - 1+ j + e 

< -2il - ma -mb)Wit) + e + 2, 



(8) 



where we use E t + 1)^] < oo for the first inequality. This shows that = (VFi(t) + 

W2it))'^ is a suitable Lyapunov function for our 'local' single-processor system under the neces- 
sary stability condition ma + m^ < 1 . 

This observation on the single-processor system motivates the following quadratic local Lya- 
punov function £ local : ^i- — ^ ^+ for the Rybko-Stolyar network: 



-^localC^;) — x^Zx 



/I 1\ 

110 

110 

Vl 1/ 



X = ixi+X4)'^ + ix2 + X3) 



for X = ixi,X2, X3, X4) 



(9) 

One can easily check that it satisfies ^ with (small) slack e > under the necessary stability 
requirements mi + m^ < 1 and m2 + m3 < 1. We propose the following global Lyapunov function 

^ global • 

£global(^(t)) = £local(VFl(t),^2(0,W^3(t),^4(t)) +e'Clocal (w^l (t) , (t) , (0 , ^^4 (t)) , 

where the new parameters ^ and VFj(t) shall be defined explicitly. We remind the reader that 
our goal is to prove ([5|). 

First, a similar calculation as for the single-processor case in ^ yields that under the LRFS 
policy. 



E 
E 



< 



Wiit + l) + Viit + l)-Wiit)-Viit) Kit) 
W^it + 1) + V2it + 1) - W^it) - V2it) I Kit) 

Hence, as for ([8]), one can conclude that for some constant C, 



.1 + mi, 



- '^W3{t)+V2{t)~i ^ 



E 



E 



iWiit + l) + Viit + l))'^ -iWxit) + Viit))^ Xit) < -2(l-mi)(VFi(t) + y4(t)) + e, 

(10) 

iW^it + 1) + V2it + l)f - iW^it) + V2it))^ I X(t)] < -2(1 - m3)(VF3(i) + V2it)) + C, 

(11) 
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where the precise value of C can be different from hne to hne. 

We note that the sum {Wi{t) + Vi{t)Y + (W3{t) + V2{t)y is not a suitable choice for £giobai 
since it does not include Q2{t) and Qi{t) (or W2{t) and W4{t)). To address this issue, we further 
use 

W2{t) := W2{t) + m2Qi{t) and Wi{t) := Wi{t) + rmQsit). 

We refer to W2 and W4 as the total workload in buffer 2 and 4, respectively. Using this notation, 
one finds that under the LRFS policy. 



E 
E 



Wiit + 1) + W4{t + 1) -Wi{t) -W^it) Xit) 

W3it + 1)+W2it+1)-W3it)-W2it) X{t) 



< 



< 



{t)+Wi{t)-l +"^1 + "14, 



LM/2(t)+iy3(i)-i + "^2 + ?n3. 



The above equalities can be used to obtain 'negative drift terms' for W2{t) and W4,{t), which 
are missing in pop and (jlip . Namely, for some constant C, we obtain 



+ e 



+ + W4(i + l)j - \ Wi{t) + WS)) x{t) 

< 2(Wi{t) + W4{t)'^ E Wi{t + 1) + W4{t + 1) -Wi{t) -Wiit) X{t) 

< -2(1 - mi - m4) {Wi{t) + Wi{t)) + 2(?ni + 7714)7714^3 (*) + 6 + 2 

< -2(1 -p^ax)(VFi(i) + ^4(i)) + 2mV^axW^3(t) + e + 2, (12) 



where pmax = niax{pi,p2} and m* = maxjj |^|- Similarly, 



E 



(W3it + 1) + W2it + 1)^ - (Wsit) + W2it)y \x{t) 

< -2(1 - p^ax) (^3(0 + W2{t)) + 2777V^axVFi(t) + 6 + 2. (13) 



Observe that there are positive terms W3{t) and Wi{t) in (fT2|) and (fT3]) . respectively. The key 
idea behind our proof is that the positive terms can be canceled out by appropriately summing 
pO|) . ([TT]) . ([12]) and (fT3|) . Indeed, we define the desired Lyapunov function £ global as 

'Cgiobai(^) = {Wi + + {Ws + Fa)' + e f + ty4V + e f VF3 + 1^2' ' 



where we choose 



2m* Pit 



Combining (UHl), (HU), ([121) and (USD, we conclude that 



E 



'C'global(-^(i + 1)) — ^g\ohaliX{t)) 



Xit) 



< -2C{1- Pra..)\X{t)\ + e. 



This completes the proof of ([5]), and hence the desired stability i^. 
4.2 Beyond the Rybko-Stolyar Network 

The preceding subsection presents the main idea behind our construction of a 'global' Lyapunov 
function ^global using a 'local' Lyapunov function ^local (i-e., from the single-processor system) in 
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the specific example of the Rybko-Stolyar network. The construction of ^global reUes on summing 
-^locai terms inductively by exploring certain maximality properties of the LRFS policy at each 
iteration. In general networks there are several difficulties which do not arise in the Rybko- 
Stolyar network, and this section discusses the ideas and arguments needed to overcome them. 

A first challenge we have overcome arises in networks with unbounded route lengths (i.e., 
mi{d G N : /"^ = 0} = oo). In that case, the above inductive procedure does not terminate. 
For this reason, we propose a variant of the LRFS policy, the e-LRFS policy, which occasionally 
processes a job with the largest counter. Intuitively speaking, this additional mechanism in 
e-LRFS can control the jobs with large counters, whereas LRFS cannot. 

A second challenge we have surmounted is that the construction of £ global in the Rybko- 
Stolyar network starts from a simple local Lyapunov function in a single-server system, but it 
is not clear whether similar arguments go through for general local Lyapunov functions and 
stochastic processing networks. We require Condition C2 to resolve this issue. It is readily seen 
that the local Lyapunov function ([9]) used in the Rybko-Stolyar network satisfies this condition. 
The condition can be relaxed under some additional conditions on the arrival processes and 
service time distributions. For example, in synchronized networks. Condition CI can be used 
instead of C2. 

A further challenge in the general case relates to the definition of Wi{t). In the Rybko- 
Stolyar network, it is the sum of workloads along a path of buffers, with i as the last buffer. 
This definition only applies to networks with deterministic routing. In the general case we use 
several notions of total workload. To allow for stochastic routing, we construct a new process 
{Y{t)} from {X{t)} with deterministic routing. This process is essentially identical to {X{t)}, 
but we enlarge the state space to incorporate routing information. We construct a Lyapunov 
function £giobai for {Y{t)}, which we use to establish the stability of {Y{t)} and hence the 
stability of {X{t)}. 

In summary, we construct the Lyapunov function £giobai for general networks as the sum of 
three parts: 

£giobai(y(t)) = E©'^i°-i(^<'=(*))+£;;ll^(*)ll2 + 4^(^(*))'- 

The specific notation used here is not important; we refer readers to Section E] for the definitions 
used. To prove stability, we need to argue that this function satisfies a so-called negative-drift 
condition. The first term, i.e., the finite sum, comes from the inductive construction under 
LRFS, appropriately truncated. For the Rybko-Stolyar network, this is the only part we need. 
The first part produces the desired negative drift for jobs with low counters, but it gives a 
positive drift in terms of remaining service requirements V{t) as a by-product (albeit not in 
the Rybko-Stolyar network under the assumptions of the preceding subsection). The second 
term in our Lyapunov function (||l^(i)||2) has a negative drift and compensates the positive 
drift incurred by the first term. The third term in our Lyapunov function (9(^(i))^) controls 
the high-counter jobs under the mechnism which is present in the e-LRFS policy but not in 
the LRFS policy (step 3 in Definition 13. 2p . This additional mechanism allows us to establish a 
negative drift for the last term. By appropriately weighing each of the three terms, we derive 
the desired negative drift condition for the Lyapunov function ^global- 
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4.3 Connection with Fluid Models 

As mentioned in the previous subsection, our approach rehes on an inductive argument based 
on job counters. For the Rybko-Stolyar network and more generahy for multiclass networks, 
fluid models can be used to give relatively simple proofs of our results. Thus, a more detailed 
discussion on the connection with fluid models together with its pros and cons is warranted. 

The fluid approach consists of two main steps. In the first step, by scaling time and space, one 
proves convergence of the queueing process to the solution of a system of deterministic equations 
known as the fluid model. In the second step, one proves that this fluid model is stable, i.e., 
that it eventually reaches the origin. Stability of the fluid model can be established through the 
construction of a Lyapunov function for the fluid model, or in some cases one can obtain fluid 
stability through direct methods such as induction. Once fluid stability has been established, 
one can apply general theorems to deduce that the stochastic model is also stable (in a certain 
sense), see for instance [3]. 

It might be possible to establish existence of a fluid model and to prove that the stochastic 
model converges to the fluid model in the setting of the present paper, and it can be expected 
that our 'global' Lyapunov function should work to prove fluid stability. Comparing our ap- 
proach with this proof strategy, a disadvantage of the fluid model is that one needs to establish 
convergence to the fluid model, while a disadvantage of our approach is that we have to keep 
track of detailed state information such as residual service times. 

Another possible approach to establish fluid stability is to use an inductive argument, which 
may seem particularly attractive given our construction of job counters and the suitability of a 
induction argument in existing work on fluid models [?]■ However, this approach has inherent 
challenges. The base step in an inductive approach could use the 'local' Lyapunov function /Ciocai 
to argue that the fluid level of jobs with counter 1 vanishes after some flnite time Ti. It would 
then use £ local to argue that the fluid level of jobs with counter 2 vanishes after some finite 
time T2 > Ti, and so forth. To carry out this argument, one has to show that Lineal satisfies a 
certain negative-drift condition under the assumption that high-priority counter 1 jobs vanish 
on a fluid scale. The latter only yields a guarantee on the 'average' or 'long-run' behavior of 
the jobs with counter 1, whereas one needs 'short-term' network state information to establish 
the negative-drift condition for jobs with counter 2. Indeed, under our scheduling policy, jobs 
with counter 1 (even when vanishing on a fluid scale) can signiflcantly influence the dynamics 
of jobs with counter 2 depending on the complexity of the network. Therefore, the base of the 
induction approach is too weak to be used in the induction step for general networks since one 
needs more detailed information than the time-average given by the fluid approach. 

In special cases such as multiclass networks, one may not need quadratic Lyapunov functions 
and it may be possible to establish the stability of our counter-based policy using fluid induction 
without quadratic Lyapunov functions. However, in general (e.g., for networks of switches), we 
need quadratic Lyapunov functions since they are the only available tool to establish stability 
for single- hop networks. 

5 Examples 

In this section, we provide applications of Theorem 13.51 to various special stochastic processing 
networks. We consider parallel server networks (including multiclass queueing networks) in 
Section 15.11 and communication networks (including wireless networks and networks of input- 
queued switches) in Section 15. 2i They are examples of non-synchronized and synchronized 
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networks, respectively. In all of these important examples, suitable local Lyapunov functions 
are easy to find. 

5.1 Open Multiclass Queueing Networks and Parallel Server Networks 

In this section, we consider special stochastic processing networks known as parallel server 
networks. These networks are characterized by the following assumption. 

Al. Each activity is processed by exactly one processor and processes exactly one buffer, i.e., 

\%j\ = l, for ah j€ a. 

Figure [2] illustrates the relations between buffers, activities and processors in parallel server 
networks. Our notion of 'parallel server network' generalizes the well-studied parallel server 
systems [11] by adding stochastic routing dynamics between buffers. It also includes open 
multiclass queueing networks [3] as a special case, which additionally require 

\di\ = 1, for all i G J. (14) 

In open multiclass queueing networks, buffers and activities are in one-to-one correspondence 
and they are referred to as classes. The Rybko-Stolyar in Section |4] is an instance of open 
multiclass queueing networks. 




Figure 2: Two examples of parallel server networks. They do not satisfy Assumption A 2. 
The leftmost diagram illustrates a parallel server network with five buffers {a,b,c,d,e}, three 
processors {1, 2, 3} and eight activities {(a, 1), {b, 1), {b, 2), {b, 3), (c, 2), {d, 3), (e, 2), (e, 3)}. Once 
a job in buffer a,b,c,d completes its service requirement, it joins buffer b,c,d,e (i.e., Pab = 
Pbc = Pcd = Pde = 1)) respectively. Once a job in buffer e completes its service requirement, 
it leaves the network. The rightmost diagram illustrates a parallel server network consisting of 
three 'local' parallel server systems where jobs are routed between local systems. 

A parallel server network naturally defines a bipartite graph (J, X, J) such that each activity 
in a defines an edge between buffers J and processors %. Requirement ([H]) of open multiclass 
queueing networks imposes the additional restriction that each vertex in J has degree one. We 
further consider the following strengthening of Assumption Al. 
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A2. (3,%, 3) is a union of disjoint complete bipartite graphs {(J^'^), DC^'^), a^''^) :he3{}, i.e., 



3^''^ = 3. 

This assumption imphes that two buffers in the same component are activity-interchangeable, 
even though they may differ with respect to routing, external arrivals or service requirements. 
One can easily check that open multiclass queueing networks always satisfy this assumption, 
while the parallel server networks in Figure [2] do not. Assumption ^4^ is useful because it 
enables us to establish a necessary condition for stability and it allows us to find a suitable local 
Lyapunov function satisfying Condition C2 of Theorem 13.51 However, Theorem 13.51 is applicable 
to general networks as long as one can find a 'good' local Lyapunov function satisfying Condition 
C2. Figure [3] gives examples of parallel server networks satisfying Assumption A 2. 




Figure 3: Two examples of parallel server networks satisfying Assumption A 2. The rightmost 
diagram illustrates the Rybko-Stolyar network described in Section HI 



Necessary condition for stability. We now aim to obtain a necessary condition for stability 
of a parallel server network. Under Assumption A 2, a necessary condition to stabilize the network 
is that for every n. 

It is clear that the above condition is required for stability since p^'^^ and (3^^^ describe the 
total nominal load and the maximum processing rate, respectively, at the local component 

Local Lyapunov function. As in Section U the single-processor example is the main building 
block. We define the local Lyapunov function as 

■i^iocai(a;) = E X] ^« for X = (xi, . . . (16) 

We now show that this function satisfies ^ with some slack e > as long as the necessary 
condition (fT5|) for stability is satisfied. For given vectors w = [w-i] € and u = [uj] € M(t(;), 
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maximality implies that, on writing w^^^ = ^jgjCh) Wi, 

where we use Assumption A2 and we recall that = 1 if x > 0, and 1^ = otherwise. Thus, 
we have 



he'K 



= e + 2 J]] Ti;^'^) (^p^'^) +em('^) - 

where C is some constant and we define m^^^ := ^^^j(h) rrii. Therefore, ^local is a- local Lyapunov 
function with slack e for 

I3ih) _ p{h) 

< e < min jr-. , 

he'K mW 

where the right-hand side is positive if (jlSp holds. 

Stability of LRFS policies. We now formulate the main results of this paper for open multi- 
class networks and parallel server networks. Under Assumption A 2, the local Lyapunov function 
(|16p satisfies Condition C2 of Theorem 13. 5[ Therefore, we obtain the following proposition as 
a corollary. We remind the reader that open multiclass queueing networks are special instances 
of parallel server networks, and that Assumption A 2 automatically holds for these networks. 

Proposition 5.1. // a stochastic processing network satisfies Assumption A2 with p^^^ < j3^^'^ 
for all n, then 



• The e-LRFS process is queue-length-stable for any e G [ 0, min — 



I3(h)_p(h) 



• The LRFS process is queue-length-stable if all routes are bounded in length. 

We note that the e-LRFS policy admits a simpler description in a stochastic processing 
network satisfying Assumption A2, since a job can be processed by any processor in the partition, 
i.e., S in Definition 13.21 is non-empty whenever a processor is idle and capable of processing a 
job. Indeed, the e-LRFS policy reduces to the following work-conserving randomized priority 
policy: whenever a processor k is idle at time t and there are jobs capable of being processed 
by k, 

• Process a job with the smallest counter with probability 1 — e ^1 — l^j^j^^^^, otherwise 
process a job with the largest counter. 

• Set = 1 if = 0, 
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where 3^^^ is the (local) component of buffers associated with processor k. Proposition 15.11 
implies that the e-LRFS policy can achieve 'almost' the full capacity region (llSh by choosing a 
small e > 0. 

Our proof of Proposition 15.11 provides a different proof for some results that have been es- 
tablished using fluid model techniques. For example, in reentrant lines, the e-LRFS policy for 
e = is identical to the well-known First Buffer First Served (FBFS) policy. Our proposition 
implies that the FBFS policy is throughput optimal in all reentrant lines, which has been proved 
originally in [7]. 

5.2 Communication Networks 

We now consider examples of synchronized stochastic processing networks described in Section 
[21 i.e., nii = f3j = 1, for all i G 3,j G 3- In particular, we consider the following additional 
assumption on synchronized stochastic processing networks. 

Bl. Each buffer has exactly one associated activity, i.e., 

di = {ji} for ah i € J. 

Hence, we write JCi := "Xj.. 

We again remark that Assumption Bl facilitates a suitable local Lyapunov function for Theorem 
13.51 However, even if Assumption Bl does not hold. Theorem 13.51 is applicable to synchronized 
stochastic processing networks as long as one can find a 'good' local Lyapunov function. Synchro- 
nized stochastic processing networks satisfying Assumption Bl include various communication 
network models of unit-sized packets: networks of input-queued switches [iHllH], wireless network 
models with primary interference constraints [21] and independent-set interference constraints 
[22j . We refer the corresponding references for detailed descriptions of the network models. 
As a concrete example, we write out the details of the wireless network model with primary 
interference constraints. 

Wireless networks with primary interference constraints. Consider a network of n 
nodes represented hy V = {1, . . . ,n} and a set of directed paths {Pi, P2, . . . }• Unit-size packets 
arrive at the ingress node of each path as per an exogenous arrival process. Assume that the 
network is synchronized, i.e., each packet departs from a node at time t S Z+ and arrives at the 
next node on its route at time t+1. The primary interference constraint means that each node 
can either send or receive (it cannot do both) one packet at the time. A scheduling policy (or 
algorithm) decides which packets transmit at each (discrete) time instance. Figure [H illustrates 
a wireless network of four nodes with primary interference constraints. 

Necessary condition for stability. As in the parallel server networks, one can obtain the 
following necessary condition to stabilize a stochastic processing network satisfying Assumption 
Bl: for ah ke%, 

p^^) := < 1. (17) 

This is because Assumption Bl implies that each buffer has at most one associated activity, and 
the processing rate is 1. 
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l->21->3 2->l 3->l 2->4 4->3 




Figure 4: Example of a wireless network with primary interference constraints with four nodes 
{1,2,3,4} and five paths {1 ^ 2, 2 ^ 1, 1 ^ 3, 3 ^ 1, 2 ^ 4 ^ 3}. There is a buffer on each 
(directed) edge on each path, i.e., six buffers in total. Unit-size packets arrive at the ingress 
buffer (i.e., the first node) of each path. In the leftmost diagram, two links 2 — ?> 4 and 3^1 are 
transmitting (unit-sized) packets. Once a packet is transmitted, it leaves the network if it arrives 
at the destination node (i.e., the last node on its path). The rightmost diagram illustrates the 
corresponding stochastic processing network, where the relation between buffers, activities and 
processors induces a hypergraph (each buffer requires two processors) 



Local Lyapunov function. We consider the following local Lyapunov function: 

^local(2;) = X] X] X] for X = (Xi,. . . ,X/). 

We now proceed toward proving that condition ([3]) holds. For given vectors w = [w,] S and 
u = [uj] € 'M.{w), maximality implies that 

Thus, we have that, on writing i/j = /3j + errii, 
^ioc<,\{w + p + em - s{u)) - Lioci,i{w) < 6 + 2^-^^ ^ ^ - E E ""^ 

i& \keX,e&:k£Xi k&XU&-k£Xi 

< e + iY^wAY. E ^^-1-. 
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where C is some constant. Therefore, ([3]) holds if u^''^ := ^i^j-k£Xi ^« ^ max gj \x-\ ^'^^ A: € IK. 
This is equivalent to 

<e < mm , ' , 

kex my'') 

where we define m^*^-* := X^^gj-^gjCi '^^^ above interval is non-empty as long as p^^^ < 
for all k e%. 

maxigj \Xi\ 

Stability of LRFS policies. We now state the main result for synchronized stochastic net- 
works satisfying Assumption Bl. Since the network is synchronized, we obtain the following 
proposition as a corollary of Theorem 13. 5i 

Proposition 5.2. // a stochastic processing network is synchronized and satisfies Assumption 
Bl with pC^) < ^ 1 ^ for all k e%, then 

maxigj \Xi\ J ' 

• The e-LRFS process is queue-length- stable for any e € I 0, min^gjc ''^ 



m 



(fe) 



• The LRFS process is queue-length- stable if all routes are bounded in length. 

Pr op osition 15 . 2 1 implies that the e-LRFS policy can achieve a ^—pri fraction of the capacity 

region (|17p . For networks of input-queued switches [181 [6] and wireless networks with primary 
interference constraints [21], it is easy to see that maxjgj = 2, and hence the e-LRFS 
policy achieves 50% of the capacity region. For wireless networks with general independent set 
constraints [22], maxjgj |3<Cj| is the maximum number of interfering neighbors, i.e., the maximum 
degree of the underlying interference graph. 

In large-scale networks, distributed scheduling schemes with low complexity have gained 
much attention recently, even though they usually perform worse than centralized ones with 
high complexity. We remark that the 50% throughput result of a greedy scheduling algorithm 
has previously been established for input-queued switches (i.e., no routing between buffers) by 
Dai et al. [6]. Proposition 15.21 generalizes this to 'networks' of input-queued switches operated 
under stochastic routing between buffers (or local switches). In wireless networks with primary 
interference constraints, Wu et al. [25] establish the 50% throughput result of the LRFS policy 
assuming deterministic routing (i.e., fixed routes between nodes), while Proposition 15.21 allows 
stochastic routing. 



6 Proof of Theorem 13.51 

To prove the desired stability of the e-LRFS process {X(t)}, we construct a new process {Y{t)}, 
which is almost identical to {X{t)}, but it has a larger state space. The main idea of the proof 
is to construct a Lyapunov function for the 'larger' process {Y{t)}, which implies the stability 
of {Y{t)} and therefore the stability of {X{t)}. 

The description of {y(t)} is as follows. Consider the stochastic processing network setup in 
Section [21 and let J" = {Pi, P2, ■ ■ ■} be the collection of all possible paths of buffers (allowing 
repetitions) of length at most D (i.e., \7\ < (/ + 1)^), where D G N is some finite constant 
to be determined later. We assume that when a job enters the network, it pre-determines the 
first D buffers on its route. After being processed from these D buffers, jobs perform the usual 
stochastic routing as described in Section[21 Let inm £ J, Qnm{t) G ^+ and Vim{t) G denote 
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the m-th buffer on patli P„, the number of jobs waiting in buffer inm and the remaining service 
requirement of the job in buffer inm being processed by activity j at time t, respectively. If 
activity j is not processing a job in buffer inm, then Vim{t) = 0. Furthermore, as before, let 
Qi,c{t) be the queue length (i.e., the number of jobs waiting for service, excluding those being 
processed) with counter c in buffer i at time t. V^^{t) is defined to be the remaining service 
requirement of the job with counter c in buffer i being processed by activity j at time t. We 
then define 

Y{t) = [Q„„(t),g,,e(t),K^^(i),^.feW>^^'^^W] ef^y:=Z-xM-x[0,l]^, 

where n, m, c, h are positive integers such that n < (/ + 1)^, m < D, c > D and h < H. 

As for {X{t)}, we impose the convention that has right-continuous sample paths. We 

define the norm through 

\Y{t)\ = Yl Qnm{t)+ Q^^-(^^+ E VL{t)+ E ^fcW- 

n<(/+l)0,m<D i&,c>D n<{I+l)0 ,m<D,je3 i&,c>D,j£g 

One can define a natural projection P such that the distribution of P{Y(t)) is identical to 
that of X(t) and |P(y(t))| = given that 1^(0) is drawn appropriately from the preimage 

P~^{X{0)) of ^(0). Intuitively speaking, {Y{t)} tosses random coins to determine routes in 
advance, and since the scheduling decisions of e-LRFS are independent of these coins, the natural 
projection of ignoring these pre-determined coin fiips provides exactly the dynamics of 

{X{t)}. Hence, it suffices to prove that the (bigger) process {Y{t) : t € M+} is queue-length- 
stable, i.e., 

limsup- / E [\Y {s)\] ds <oo, (18) 

for any given initial state Y{0) £ fiy := {Y € ^Iy ■ \Y\ < oo}. In essence, this follows from the 
following proposition, which is proved in Section 16.11 

Proposition 6.1. If the conditions of Theorem \3.5\ hold. then there exist constants N, T, D,(,Q ^ 
(0, oo) and a Lyapunov function -Cgiobal '■ ^+ U {oo} such that for all t > N , 

E[L^l,^,l{Y{t + T)) \ J{t)] < £giobal(l^(t))-Cl>^(t)l + e, almost surely, 

and supygQ*^ ■i^giobal(^)/|^P < oo, where we define the filtration {3'{t) : t > 0} by 

J(t) := a{Y{s) :0<s<t}. 

Now we describe how Proposition 16.11 implies (jlSp . and hence the conclusion of Theorem 13.51 
First one can observe that E [|y(t)p] < oo since Y{0) € assume bounded second 

moments on arrivals and service times. Since supyg^^*^ £giobai(^)/|^P < oo from Proposition 
16.11 it follows that 



E[j:g,oh.i{Y{t + T))],E[Lgiob,,{Y{t))],E[\Y{t)\] < oo. (19) 
Combining Proposition 16.11 and (I19p yields that for all t > N, 

E [£giobai(>^(i + T))] < E [£giobai(i"(i))] - (E [\Y{t)\] + 6. 
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Therefore, we have that for t > N, 

f-T+t ft 



Jt+n Jn 

< f E [£giobai(>^(5))] ds-C f E [\Y{s)\] ds + Q{t - N), 

JN JN 



which imphes that for t > T + N, 

rT+N 



c 



t-N 



ft 1 rJ+JV 

/ E[\Y{s)\]ds < e + —— / E[L,,,UYmds. 

JN t - I\ Jj^ 



The right-hand side of the above inequahty converges to C as t — )■ oo. This leads to the desired 
conclusion 

6.1 Proof of Proposition 16.11 

The choice of N in Proposition 16. II comes from our assumption on the external arrival processes 
in Section [21 namely, that there exists some N < oo such that for alH G J and t > N, 



E 



Ai{t, t + 1) \ {Alio, s),..., Alio, s):0<s<t} 



< Oli. 



For notational convenience, we assume = in the proof of Proposition 16. 1[ Namely, we 
assume that for all t > 0, 



E 



Aiit,t + l) \'Jit) < ai<oo. (20) 



All the proof arguments are applicable to the general case > 0. 

We first define some further notation in Section [6TLTJ The skeleton of the proof of Proposition 
I6.1l is described in Section [5.1.2| and it uses three key lemmas. The proofs of these lemmas follow. 

6.1.1 Notation 

The quantities defined below are simple functions of the network state Yit). We use bold 
symbols to denote vectors of quantities, e.g., Qdt) = [Qi,cit)] and VF<c(i) = [M^j,<c(i)]- 

Queues. For c G N, let Qi^dt), Qi,<cit) and Qi^<cii) be the number of waiting jobs with 
counter c, < c and < c in buffer i at time t, respectively. That is, we set 

Qi,c(i) = ^ Qncit) I{i}iinc) ioT:c<D, 

n=l 

c— 1 c 

Qi,<cit) = '^Qi,c'it) Qi,<cit) = Qifi'jt), 

c'=l c'=l 

where we recall that /{j}(z„c) = ^ inc = i and /{j}(inc) = otherwise. Furthermore, for c < D, 
we let Qi^<cit) be the number of waiting jobs with counter < c 'in or destined for' buffer i at 
time t. Namely, 

(I+l)^ c c 

Qi,<cit) = Qi,<cit)+ Qnmit)I{i}iinr), 

n=l m=l r=m+l 
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where the first term '(5i,<c(i)' and the second term • • • ' on the right-hand side count 

the numbers of waiting jobs currently in buffer i and destined for buffer i, respectively. 

Workloads. Let V^{t) denote the remaining service requirement of the job being processed by 
activity j at time t, and Vi{t) the total remaining service requirement of the jobs being processed 
in buffer i at time t (multiple jobs can be processed from the same buffer by different processors). 
Similarly, Vi^dt), l^,<c(i)) ^,<c(i) stands for the total remaining service requirement of jobs with 
counter c, < c, < c being processed in buffer i at time t, respectively. We furthermore define 
the following quantities: 

if the network is not synchronized, 



Wi,<cit) = miQi,<e(t) + V-(t) 
Wi,<cit) = miQi^<c{t) + Viit) 
Wi^<cit) = miQi,<c{t) + Viit). 



if the network is synchronized. 



Wi,<c{t) = miQi^<c{t) + Vi,<c{t) 



We use different definitions for these variables depending on whether the network is synchronized 
or not, since the strategy of the proof differs in each case, e.g., see Section [6.1.4[ We also note 
that in a synchronized network, Vi^cit),V^i,<cit),Vi^<c{t) ^ {0)1} for t £ The quantities 

Wj^<c(t) and Wi^<c{t) are (expected) immediate workloads, since they only involve work that 
is in buffer i at time t. The quantities Wi^<c{t) are (expected) total workloads, since they 
incorporate work currently in the system which will be routed to buffer i, regardless where the 
work resides in the network at time t. 



Three types of jobs & v^eights. We distinguish three types of jobs: 

Type 1. Jobs with counter > D 

Type 2. Jobs on a path of length = D m y. 

Type 3. Jobs on a path of length < Z) in CP. 

Hence, the counters of jobs of Type 2 and 3 cannot exceed D. We further note that each 
job (regardless whether it is currently being processed or not) can compute its expected total 
remaining service requirement in the future (under the network process {Y(t)}), and we call this 
quantity the weight of the job. Namely, if a job is not currently being processed, its weight is 

nii E [the number of times it visits buffer i before it leaves the network] , 

which includes the current buffer of the job. Thus, the weight of a waiting job, not currently 
being processed, can be calculated as follows. 
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Type 1. The weight of a waiting job of Type 1 in buffer i € J is 

eJ{I + P + P^ + ---)m = eJ{I-P)-^m, 

where m = [rrii] G M;?^ and S {0, 1}^ is the unit vector with zeros except for its 
i-th coordinate which equals to one (both are column vectors). 

Type 2. The weight of a waiting job of Type 2 in buffer inm on a path P„ of length D (i.e., 

\Pn\=D) is 



D 



D 



k=m k=m 

where we note that ino is the last buffer on P„. 
Type 3. The weight of a waiting job of Type 3 in buffer inm on a path P„ of length < D is 



E 

k=m 



m 



On the other hand, for jobs currently being processed, each weight is calculated as 

the current remaining service requirement 

+ rrii E [the number of times it visits buffer i before it leaves the network] , 

where the latter number does not include the current buffer of the job. Now let Mi(t), M2{t) 
and M3(t) be the total weights of jobs of Type 1, 2 and 3 at time t, respectively. These total 
weights are fully determined by the network state information Y{t): 



Mi{t) 

M2{t) 



Q^,c{t) ef (/ - P)-^rn + (t) Mc(t} + ef P(/ - P)-'m 



i&,c>D 



n<(/+l)0,m<D:|P„|=D 



i&,c>D,je3 
el^P[I-Py^m+Y^ 

k=m 



D 



+ 



E 



n<{I+l)0,m<D,j£j:\P„\=D 



D 



V^^{t)+el^PiI-P)-'m+ Y 



m 



Ink 



k=m+l 



E 

n<{I+l)0 ,m<D:\P„\<D 



\Pn\ 



k=m 



+ 



E 



n<(I+l)0 ,m<D,je3.\Pn\<D 



\Pn\ 



m. 



k=m+l 



where we recall that 1^ = 1 if x > 0, and it = otherwise. 
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6.1.2 Three Auxiliary Lemmas and Proof of Proposition 16.11 

We state and prove the following three key lemmas, which we prove in Section 16.1.31 16.1.41 and 
16.1. 5^ respectively. To simplify notation, we will use C to denote a finite constant which only 
depends on the matrix Z given in Theorem 13.51 or on the predefined network parameters from 
Section [2j Its precise value can be different from line to line. 

Lemma 6.2. There exists a constant C < oo such that for all t > 0, 

E[\\V{t + l)\\l\m] < \\V{t)\\l-(3^i^\\V{t)\\i + e, 
where /3mm = minjgg/3j. 

Lemma 6.3. Suppose that there exists a symmetric matrix Z S M;^^^ such that £iocal(2^) = Zx 
is a local Lyapunov function with slacks > 0, and that either Condition CI or C2' from Theorem 
\3.5\ holds. Then given Z) € N, there exist constants v,Q = C{D) G (0, oo) such that for all c < D, 
t > 0, 

E 



£local [W<c{t + 1) j 

< £iocai (w<cit)) - v\\Q<M\i + e (IIF(t)iii + ||Q<,(t)||l + 1) 



Lemma 6.4. Consider e > 0. // the network is synchronized or if Condition C2 from Theorem 
13.51 holds, then there exist constants D,T (^f^ and Q, 71, 72 € (0, 00) such that for all t >0, 

E[9{Y{t + T)f I < g(y(t))2_^2||Q>DWIIi + e(||Q<D(t)||i + ||F(0l|i + l), 

where g(y(t)) := Mi{t) + A^hit) + ^i\\Vit)\\i. 

Lemma [6. 41 is not needed for the proof of Proposition 16. II if all routes are bounded. Hence, for 
networks with bounded routes, e = is allowed and only Condition C2' is needed for the desired 
stability. The right-hand sides of the inequalities in Lemmas 16.21 - 16.41 provide 'negative drifts' 
on ||V(t)||i, ||Q<£)(t)||i and ||Q>_D(i)||i, respectively. By appropriately weighing the functions 
in these three lemmas, we shall construct an appropriate (global) Lyapunov function global- To 
this end, from Lemma [6^ we obtain 



E[\\V{t + T)\\l I :J(i)] < E[\\V{t + T-l)\\l I J{t)] -(3^,^E[\\Vit + T-l)\\^ \ + C, 

< E [\\v{t + T-i)\\l I j(t)] +e 

< E [\\V{t + T-2)\\l I J(t)] +26 

< E[\\v{t + i)\\l\m]+{T-i)e 

< \\Vit)f2-Pmin\\V{t)\\i+Te, (21) 

where T is the constant from Lemma 16.41 We write this inequality as 

1 



E 



^—\\V{t + T)g J(t) 



< ^\\Vit)g-\\V(t)h + e, (22) 
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where the constant C is redefined appropriately. We now argue that, similarly, Lemma [67 
implies that 

E 



£local ( VF<e(t + T) j 

< Aocaifvr<,(t)) -z;||Q<,(t)||i + e(||F(t)||i + ||Q<,(t)||i + l), (23) 



where C is some (large) constant which may different from the one in Lemma 16.31 To see why 
this holds, we use the same argument that led to ([2T]) with the additional observation that, for 
some constant C, 

sup {E[\\V{t + s)\\i\m]-\\V{t)h) < e', (24) 

s6{0,lv,r} 

sup {E[\\Q{t + s)\\^\m]-\\Qmi) < e'. (25) 
se{o,i,...,T} 



Here (I24p can be derived along the lines of (I2ip . and (I25p follows from (I20p and the observation 
that the change in queue length is majorized by the number of external job arrivals. 

We now show that Proposition 16.11 follows from these three lemmas, where the last lemma is 
not needed if all routes are bounded. We consider the following Lyapunov function ^jgiobaV- 



If all routes are bounded, 

D 



£global(y(i)) := E(^)'^l°-l(H^<c(i))+^||Fr^^ll? 

c=l 

where we choose D < oo such that = 0, and v comes from Lemma 
• Otherwise, 

£giobai(i^(t)) := E(^)'^io-i(^^^<,(t)) + ^||F(t)||i + ^g(y(t))2, 

where v and D is from Lemma [673] and [67il respectively, ^ := ^{v, C,D) = v (^)^ and C 
is a large enough constant chosen so that it can be used for Lemma [6.41 as well as for (|22p 
and ([23D. 



2) 



We focus on proving Proposition 16.11 for the case of unbounded routes, but all arguments go 
through for the other case. Without loss of generality, we assume that 

^,17,72 < i< e. 

The property supygf^*^ '^giobai(^)/|^P < oo in Proposition 16. II is readily seen to hold. To derive 
the negative drift property, we observe that Lemma [67¥[ in conjunction with ([22]) and ([23]) imply 
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that 



E global (5^ (* + - L 

global 



D 



<e* -2e\\v{t)\u + Yl 



c=l 



V2e 



t;||Q<,(t)||i + e||Q<,(t)||i + e||F(t)||i 



' 26 



Q>Dmi + u\\Q<Dmi + \\vmi + i) 



D 



< e* -e\\v{t)\\i + Y, 



c=l 



(- 

V2e 



u|lQ 



<c 



1 + e||Q 



-^IIQ>D(i)lli + f (IIQ<dWIIi + Ill-will + 1) , 

where C* is some (large enough) constant and we use v < Q for the last inequality. The sum in 
this expression can be bounded as follows: 



D 

E 

c=l 



(- 



v\\Q 



<cl'';ili 



+ e||Q 



<c 



D 



D 



c=l 



26 



D 



2 ^ V26 

c=l 



c-1 



\Q 



<c 



c=l 



D 



< -^[^) \\Q<Dimi 



-^\\Q<D{l^)\\l- 

After combining the preceding two displays, we obtain the desired negative drift property: 



E global (^(i + T)) - L 

global 

i2i 



2g uQ>Dmi + \[\\Q<D 



< 6*-6||F(t)||i-^||Q<^(t)||i 

1 + \\Q<Dit)h + IIQ>Di'';iii 



+ \\v 



+ 1 



= e* + |-^(||F(t)||i + ||Q(t)||i), 

where we use (,,v,^2 < 1 and 6 > 1. This completes the proof of Proposition 16.11 



6.1.3 Proof of Lemma [672] 

Recall that ||T^(i)||2 is the sum of squares of the remaining service requirements of jobs being 
processed by some activity at time t, i.e., 



Y^V'itf and \\V{t)\\, = J2^^{t). 

je3 
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On the event {V^{t) < activity j has to restart before time t + 1 and hence, 

E[V^{t + lf I J{t)] 
< E [V^{t + 1)2 I V^{t + 1) > 0, 3"(t)] 

r? m I y^(t + i)>o,:j(t),j(*+i)' 



< E 
= E 



E 



V\t + l)>Q,^{t) 



where we let Ti_^^t+i denote the service time generated by the job being processed by activity j 
at time t + 1 and we write 

for the time when this job starts its service. Note that, again on the event {V^{t) < /3j}, 



E 



r? I w(t + i)>o,:J(t),j(*+i) 



Pr 

oo 

Pr 

oo Pr 



dx 



Pr [Ti^ >Pj + J(m))] 



dx 



dx 



< 



Pr 



Tl > X 



Pr[r,, >/3,] 



dx 



where we recah that P j . stands for a generic service time for buffer ij . We have thus estabhshed 
that on {V^{t) < (3j}, ' 



(/3j)2 otherwise 



(26) 



On the other hand, on the event {V^ {t) > f3j} 



yi(t + l)2 = (yi(t)_;3^.)2 < YJ(^tf_p^Yj(^t) + p]. 
Hence, combining (j26p and (j27p . we find that for some constant C < oo, 

E[v^{t + 1)2 1 j{t)] < v^{tf - pjV^{t) + e, 

which leads to the desired conclusion of Lemma 16.21 



(27) 
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6.1.4 Proof of Lemma [673] 

For notational convenience, we stick to the case t = in the conclusion of Lemma 16.31 Namely, 
we show that 



^locai Vr<e(l) I J(0) < £iocai W^<c(0) -t;||Q<,(0)||i + e(||V(0)||i + ||Q<,(0)||i + l) 



E 



However, all arguments go through for general t > 0. 

Non-synchronized network. We first consider the case when the network may not be syn- 
chronized. The first step in the proof is the observation that the schedule <T(t) under the e-LRFS 
policy is always maximal with respect to the vector w = [wi] = [miQi^^cit)] for any c > 1. Con- 
sequently, since the local quadratic Lyapunov function ^locail^^) = Zx satisfies ([3]), we obtain 
that for t G [0, 1], 

2 ^ Ziiwe i Pi + errii I3jaj{t) \ < -r]\\w\\i + e, 
where we further use the observation that 



Wi 



+ Pi + errii - ^ Pjcrjit) {wi + pi + errii - ^ Pj(^j{t) - WiWe 



< Wil Pi + errii l^j^ji^) \ +Wi\pi + erni - ^ /3jcrj{t) + 6. (28) 
V ieg; / V jG3i J 

We remind the reader that we write C for a finite constant which may differ from line to line. 
Since we set Wi = miQi^^cit), it follows that 

2 ^ ZiimiQi^<c{t) \pi + emi-^Pjaj{t)\ < -ri^miQi^<c{t) + e. 

After taking conditional expectations given 3"(0) on both sides in the above inequality, we obtain 



2 ^ Ziimi- E 



Qe,<c{t) j Pi + emi - ^ /3jfTj(t) 
j^3i 



m 



< -r/^m,Q,,<e(0) + e. 



where we use that E[Qi^<c{t) \ 9"(0)] > Q^,<c(0) — C for t € [0, 1] for some constant C. This 
can be verified by suppressing any arrivals and letting all activities work, and then using the 
standard fact from renewal theory that any renewal function is finite. Similarly, we obtain from 



E 



Qe,<c{t) I Pj<^j{t),3'{0) > Qi,<M - e' for t G [0, 1] that 



2 ^ ZiimiQe^<c{0) ■ E 

{i,e)&x3 



Pi + emi - ^ /3j(Tj(t) 



< -r]Y,miQi,<M + Q, (29) 
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where the constant C again has to be redefined appropriately. We leave this inequahty for later 
use. 

The second step in the proof is to bound E Wj^<c(l) — Wj,<c(0) 5'(0) for fixed i. Let ^i,<c 
be the number of job arrivals contributing to an increase in (5i,<c(') during the time interval 
[0, 1). Then, one can check that for A<c = ^i,<c 



E 



A<c I :?(0) < (/ + P + • • • + P") a < A, 



since we assume (pOj) . Define the following quantities. 

• Di^<ic and Di^^c are the numbers of jobs in buffer i which start their service during the 
time interval [0, 1) and with counter < c and > c, respectively. 

• ^i,<c and i?i,>c are the total amounts of service times generated by jobs contributing to 
Di^<ic and Di^^c, respectively, again during the time interval [0, 1). We stress that the 
contribution of each job to these quantities may exceed the service time it receives during 
the interval [0, 1). 

• R-l^y^ are the total amounts of service times generated by jobs contributing to Di^yc due to 
the LRFS policy (i.e., step 4 in Definition 13. 2|) . In addition, we set -R-^^^ ~ Ri,>c — R^i>c- 

Then, we have that 

Ai.<c := Wi,<c{l) -Wi,<M = mi (li,<c- A.<c) +P*,<c + Pi,>c- V / l3jajit)dt. (30) 



Taking conditional expectations given 3'(0) on both sides, we obtain 
E [A,,<, I J(0)] 

< ruiXi + E [-miDi^<c + Ri,<c \ ^(0)] + E [Pi,>, | J(0)] - E 







[ f3jaj{t)dt 


m 


Jo 





Pi + E [P,,>, I J(0)] - E 







I I3jaj{t)dt 


m 


/o 





(31) 



Now we bound E [Pi,>c | 3"(0)] in the above inequality, or equivalently E 



(L) 



t,>c 



m 



+ 



E 



t,>c 



3"(0) since Ri^yc = Ri^yi + ^f^c- First, one can check that 



E 



R. 



(M) 
i,>c 



< errii . 



This is because the expected number of jobs in buffer i which start their service during the 
time interval [0, 1) due to step 3-1 of the e-LRFS policy in Definition 13.21 is at most e since 
each timer T^^^t) is zero at most once during this time interval. On the other hand, to bound 

3'(0) , consider activity- interchangeable buffers i and i (which includes i = i). Let £,£ 



E 



R) 



i,>c 



be the event that every job in the queue Qe.<ciO) (i-e., jobs with counter < c waiting in buffer 
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(. at time 0) starts service during the time interval [0, 1]. One can observe that -Rj-^^^ 
complementary event fi^. We let 



on the 



N 



}i,<M~\ii\ 



U)=l 



where {r^ ^,} are identical random variables with mean and variance ?| < oo. We first 
consider the case when 



(32) 



Since £^ occurs only if at least Q£,<c(0) — jobs complete their service requirements, we obtain 
the following on the event that (j32p holds: 



Pr[££ I J(0)] < Vt[N< /3niax|a^| | H^)] 

= vy[n- e[n I Qi,<cm < /Jmaxia^i - E[N I Q,,<cm I s'm 
?|(Q^,<c(o) - lad) 



Af-^[iV I Q£,<c(0)])' > {E[N I Q^,<c(0)] -/3n,ax|ad)' I HO) 



< 



< 



{E[N I Q£,<c(0)] -Anaxla^!) 

^|(Q£,<c(o) - lad) 

(m^(Q^,<c(o)-iad)-/3maxiad)' 
e 

Qe,<M + l' 



(33) 



where /3max = maxjgg /3j, S is some (finite) constant depending on c;^, mi, \Se \ and we use Markov's 
inequality in conjunction with (j32p . On the event that (j32p does not hold, i.e., when Q£,<c(0) is 

bounded above by |a^| f /3max + ^) ) oiis can redefine the constant C so that (i33]l holds. Hence, 



33j) always holds. 
Using (|33]l . it follows that 

E[Ri,y, I J(0)] 



(M) 

'i,>c 



+ E 



R 



(L) 

'i,>c 



m 



< em, + Pr[£^ | J(0)] • E 4% \ J(0) 



< errii + 

< errii + 



Q£,<c(0) + 1 

e 

Q£,<c(o) + r 



where the last inequality requires that the constant C has to be redefined appropriately, since 



^i^>c I S^,!5"(0) < C as can be seen using arguments similar to those leading up to (i26]l . 



E 

Together with ([3T|) . this leads to 

E [Ai,<e I ^(0)] < pi + evii + 



Qi,<M + 1 



E 







[ (3jaj{t)dt 


m 


JO 





(34) 
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for any activity- interchangeable buffers £ and i. 

The third step in the proof for the non-synchronized case is to prove the conclusion of Lemma 
6.31 By a similar argument as in (j28p . the claim follows with v = rymiujgj mj after we show that 



2 Y %<M E[Ai^<c I no)] < -vYl ^iQi,<M + e ^ [vm + Qi,<M + 1] > (35) 



where we use that 



E [Ai,<eA^,<e I :j(o)] < e 



for all i, £ and some constant C. To see that p6p holds, it suffices to show that E 
C by the Cauchy-Schwarz inequality. This can be shown using ([30|) and 

E 



4<c I ^(0) 



(36) 

< 



Al^, I m] ,E[Dl^, I 3-(0)] ,E[R^ I :T(0)] < C 



where these bounds can be derived using arguments similar to those leading up to (j26]) . Since 

Zie,E[A,^<^ I J(0)] < e and 

t^^^,<c(0) < W^£,<c(0) + J^miQi,<e(0) = m^Q£,<,(0) + y£(0) + ^miQi,<,(0), 



the inequality in (j35|) reduces to 



2 ^ Zi£m£Q^,<c(0)^[Ai,<e I :r(0)] < -r7^miQi,<e(0) + e. (37) 



We prove this using (|29p and (j34p in conjuction with Condition C^' as follows: 

2 ^ Zi^m^Q^,<e(0)£;[Ai,<e I :j(o)] 



< 2 ^ Zj£ miQi^<c{0) Pi + errii + 











E 




m 


) 



















E 




m 


j +e 



< 2 ^ Ziim£Q£^<c{0 

< -r?^miQi,<c(0) + e, 

where we again remind the reader that the constant C may differ from line to line. This completes 
the proof of Lemma 16.31 for non-synchronized networks. 



Synchronized network. Now we consider the case when the network is synchronized, i.e., 
Condition CI. We establish the same three steps as in the non-synchronized case. In the non- 
synchronized case, we used the fact that the schedule cr{t) under the e-LRFS policy is maximal 
with respect to [miQi^^dt)], for which we required Condition C2'. In synchronized networks, as 
a first step in the proof, we use a different (stronger) maximality property, which allows us to 
relax Condition C2' to Condition CI. To this end, we introduce some necessary notation. We let 
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Cj,<c{t) = 1 if activity j processes a job with counter < c at time t (and (Jj^<c{t) = otherwise). 
Since (Tj^^dt) = c^j,<c(L*J) in synchronized networks, we write crj^<c = f^i,<c(0) = (yj,<c{f) for 
t € [0,1). The main maximaUty property we use in synchronized networks is that, under the 
LRFS pohcy, the schedule [cj,<c-fj('i) (^j)] is maximal with respect to [Qi,<c(0)/j(h) (i)] for each 
component J^'^^. Together with ([3]), this implies that for every partition 

(38) 

where we let Slrfs denote the event that at time the e-LRFS policy for component J*^^^ does 
not select a job for processing through step 3-1 (see its description in Definition 13. 2p and all 
selected jobs are due the LRFS policy in step 4. We stress that in synchronized networks, every 
processor completes the service requirement of the job it processes at every integer time t G Z+ 



and hence Pr 



p{hj) 



1 — £. Inequality ([38]) is analogous to (p9|) . which concludes the first 
step in the non-synchronized case. 

We proceed with the analog of the second step from the non-synchronized case, i.e., bounding 
E Wj,<c(l) - VFi,<c(0) I ^(O) . We stress that the definition of Wj,<c(t) differs from the one 
used in non-synchronized networks, see Section I6.1.1[ Scheduling decisions are only made at 
integer time epochs (i.e., a{t) = a{\t\)) in synchronized networks, so that 



E 



w^i,<c(i) - Wi,<M I :j(o) 



= E \Ai,<, I J(0)] -Y^E [a,- <e 



where we again let Ai^<_c be the number of job arrivals contributing to an increase in Qi^<c{') 
during the time interval [0, 1). On writing Aj^<c := VFi^<c(l) — Wj,<c(0), we have 



E [Ai,<, I J(0)] < Pi-Y^ [^i.<- I ^(0)] • 

ie3i 



(39) 



The third step in the proof for the synchronized case is to prove the conclusion of Lemma 
As in ([37j) . it suffices to prove that 

2 Yl rn^Zi^Q^^<ME[^i,<c\HO)] < -r/ m,Qi,<e(0) + C. 



Since rrii = \ and 1^(0) < 1 in synchronized networks, this reduces to 

2 Y ^i£Q£,<c(0)^[A,,<, I J(0)] < -r]YQi,<M + Q- 

Combining (pQl) and (f40l) . it suffices to show that 



(40) 



2 Y Qi,<M [pi-Y^ [^i'<- 1 ^(0)] ^ E + e. (41) 
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For activity j € d, writing 3^^^^ for the component of buffer ij (i.e., ij G 3^^^^), we have 



E [aj,<c I > Pr 



E 



aj-<e I J(0),£ 



> E 

Therefore, ()4ip fohows after arguing that 



(7,-<, I J(0),8i 



LRFS 



The above inequahty follows from (I38p and Condition Ci. This completes the proof of Lemma 
for synchronized networks. 



6.1.5 Proof of Lemma 16.41 

For notational convenience, we again restrict attention to the case t = in the conclusion of 
Lemma [131 namely, we show that for some Z),T, 6,71,72 € (0,oo), 

E[^{Y{T)f I m] < g(y(0))2-72||Q>^(0)||i + e(||Q<^(0)||i + ||y(0)||i + l), 

where we recall that 

g(y(t)) = Mi(t) + M2(t) + 7i||v(t)||i. 

All arguments are applicable for general t > as well. First observe that M(t) := Mi{t) + M2{t) 
can only change through the following events for jobs of Type 1 and 2. 

Arrivals. M{t) increases when new external arrivals of Type 2 occur. Note that there 
are no such external arrivals for Type 1. 

Routing. M{t) may increase or decrease when a job with counter > D (i.e., Type 1 or 
2) is routed since the weight (i.e., future workload) of a job conditioned on the buffer to 
which it has been routed is different from the (unconditional) weight before it is routed. 
However, M{t) does not change when a job with counter < D (i.e.. Type 2) is routed since 
it is routed deterministically. 

Starting Service. M{t) may increase or decrease when a job of Type 1 or 2 begins 
service, generating its service time at this point. Assuming the job is served from buffer i, 
then M{t) increases if the random service time is larger than its mean nii, and decreases 
otherwise. 

Being in Service. M{t) decreases when a job of Type 1 or 2 is currently being processed. 
Now we express M(t) as follows: for t G N, 

M{t) = M(0) + Marrival(i) + Mrouting(t) + Ms.servicc(i) + Mb_scrvice (*) , 
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where Marrivai(i), Afroutmg(t), Ms_service(i) and Mb_scrvicc (*) describe the change in M{t) in the 
time interval [0, t] due to events of new arrivals, routing, starting service and being in service, 
respectively. Hence, 

Ma„ival(t) > and Mb -service 
Prom our definition of the weights, one can further observe that 

-serviee 

oo 

^[M™i(t) I < tY,dm^^^\\P'^a\\i, (42) 

d=D 

where we define rn-max := maxjg^mj. 

By appropriately defining constants 7,T, D,7i,72 G (0,oo), we first prove the following. 

E[9{Y{T))-9iYm\m] < (7 ^^^>^.(°)^°, (43) 

I c otherwise 

for some constant C < 00. It is not hard to prove (j43p for Q>£)(0) = 0, and hence we only provide 
the proof for <5>£)(0) 0. Consider the two complementary events: ||T^(0)|ji > 2Ji?i.enewai and 
||V(0)||i < 2Ji?i.cnewaii where -Brenewai IS some constant which will be determined later. 

First case. On the event {||V^(0)||i > 2Ji?i.enewai}> we observe that ||V(t)||i = J^jes'^'^i^) 
and 

[ ^renewal Otherwise 

where one can find an appropriate constant ^renewal < 00 depending on the variances of the 
generic service times {Fj} using the renewal theory (e.g., -Brenewai = ^SiXi^j E[Tf]/ E[Ti] from 
the proof of Proposition 6.2 in [1]). Hence, in case there is no j G ^ satisfying (0) > PjT, 

E[\\v{T)\\,\m] = j;i?[F^'(r)| J(0)] 

— -^renewal 

< ||V(0)||i- J^renewal, (45) 

since ||V^(0)||i > 2Ji?i.enewai- On the other hand, if there exists a jo £ 3 satisfying V^°{0) > PjqT, 
we have 

E[\\v{T)h\m] = J]i?[F^'(r)| J(0)] 

= E [V^°{T) I J(0)] + J2 ^ [V^T] I 3-(0)] 

i6g\{jo} 

< Wo(0)-/3,,T+ max{W(0)-/3,r,Srenewai} 

iG3\{jo} 

< Wo(0)-/3,„r+ Yl (^'(0)+Srenewal) 

ie3\{io} 

< ||V(0)||i + JBrcnewal-/3minT 

< ||V(0)||i - J^renewal, (46) 
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where we use and choose 

2 J By-cx-^cwal 



T > 

Therefore, in both ([I5]) and ()36|) . we have 



+ 1 



/^min 

E[\\V{T)\\i\J{0)] < ||F(0)||i- JZ?,enewal. 



(47) 



Using this, it follows that 

E[9{Y{T)) - g(y(0)) I 3-(0)] < E [M{T) - M{0) \ J(0)] + jiE [\\V{T)\\i - ||y(0)||i | J(0)] 

< E [Ma„ival(r) I J(0)] - Tl^^rcnewal 

oo 

d=D 

< -7, 

where we use ()42p and now define D,-yi in terms of 7 as follows: 

27 



JB 



renewal 



D = D{j) := min <^ x G N : T^dm,,,.^^ < 7 [ 7i = 7i(7) := 

I d=x J 

We specify the value of 7 at a later stage in the proof. This completes the proof of (I43p on the 

first event {||F(0)||i > 2JSrenewal}. 

Second case. Now consider the second event {||V^(0)||i < 2Ji?i.enewai}- From our choice of T 
in (|47l) . we have 

i63 /3j /?min /^min 

which implies that before time T — 1, all activities have completed serving the jobs they were 
serving at time (and they could have worked on other jobs as well). Thus, it follows that 

E[\\V{T)\\i I J(0)] < Ji3,e„ewal. (48) 

Consider a job with the largest counter (and hence, contributing to Q>£)(0)) at time and 
let J^''"'') c J be the component the job belongs to. Define the event £step 3-i 

that the e-LRFS 

policy executes step 3-1 (see the description of e-LRFS in Definition 13. 2p for this component 
at least once before time T — 1. Let £*tep 3_i be the subevent that the job identified in step 1 
of the policy is selected for processing when step 3-1 is carried out for the first time, so that 
Pr[£step 3-1 I Sstep 3-i] = On the event 

^step 3-1' random variable j* denote the activity 

which is chosen to process this job, and let X* be the associated service time. We then have 
that = rrii.^. Observe that 

Em -service 

(T) + Mb 

-service 

(T) |i*,X*,£* 3-i,W] 



< 



-iTii , +X* - f3j, on the event {X* > } 



-irij „ otherwise 
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It thus follows that 



-service 

(T) + Mb 

-service 



< E 

< E 
= E 



-mi 



-rrii 



+ [X* - f3. 



^stcp 3-1 



.m 



E 



-m. 



+ [X* — /3min]-|- I £*tpp 3_ 



^stcp 3- 



< max E [-rrii + [Fj - /3min]+] , 



where Fj is a generic service time for buffer i and = x if x > and [x\j^ = otherwise. It is 
easy to see that E [—uii + [Fj — /3inin]+] < for all i since E\r.j\ = m.j > and /3niin > 0. Hence, 
the above inequality implies that 



with 



-service 

(r) + Mb -service {T) C'Step 3-1; 



V := min£; [mi - [Ti - /3min]+] > 0. 

i€3 



(49) 



If the network is synchronized or if the second part of Condition C2 of Theorem 13 . 5 1 holds . the 
only way for the event £step 3-i not to occur is that there exists a processor (for the component 
jC^o)^ processing a job constantly during the entire time interval [7^^°\0),T — 1] (i.e., the job 
starts service before time 7^'^'^\0) and is still in service at time T — 1). Recall that before time 
T — 1, every processor completes the service requirement of the job it was processing at time 0. 
Hence, in addition to (j47|l . if we choose T to also satisfy 



T > 2 max mi + 2, 



then it follows that 



Pr[£step 3-1 I 9"(0)] > 



K 



(50) 



where we use the fact that if a processor starts to process a new job in the time interval 
(0,T('^»)(0)] C (0,1], its service requirement is at most 2maxjgjmi with probability 1/2 by the 
Markov inequality. From ()49p and (jSOp . we conclude that 



E[M,. 



.(r) + Mb -service 

(T) I 3-(0)] < 



(51) 



From (|42]), ([48D and ([ST]), the desired inequality (j43l) follows as 

E[9{Y{T)) - g(y(o)) I m] 

< E[M{T)-M{^) I 3-(0)]+7i£;[||F(r)||i-||F(0)||i | 3-(0)] 

< E [Afarrival(r) | J(0)] + E[M, -service 

(T) + Mb -service 

(T) I 9"(0)]+7iJSrcnewal 



< r ^ dmmax 

d=D 

< -7, 



|-f"^0!||l ~ |f + 7l<^-Si.cncwal 
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where for the last inequahty we define 

ev 



7 := 



2K+2- 



Here we note that D = D{'y) < oo as long as 7 > (i.e., e > 0). This completes the proof of 
I3l) on the second event {||T^(0)||i < 2 JSrcnewai}- 



Completing the proof of Lemma 16.41 Now (H3]) implies that 

E[9{Y{T)f - 9{Y{0)f I no)] < 2g(y(0)) E[S{Y{T)) - 9{Y{0)) \ J(0)] + C 

< 2 (-7II0. .(mih + e f 1 - Ito. .rmih )) 3(^(0)) + C 



IIQ>d(o)||i ' " V IIQ>d(o)IIi 

= 2 (-(7 + 6)1+3^^(0)11. + e)s(no)) + e 

where one can check E (S(y(T)) — S(y(0)))^ | 9"(0) < 00 for the first inequality and the 
precise value of S can be different from line to line. Finally, we define 

72 := rUjainl 

and the conclusion of Lemma 16.41 follows from 

E[9{Y{T)f - 9{Y{o)f I j(o)] < 2 (-(7 + e)i+Q^^(o)||^ + e) g(y(o)) + e 

< -27m„in||g>B(0)||i + e (||Q<z)(0)||i + ||F(0)||i + 1) 

= -72||Q>d(o)I|i + e (||Q<z5(o)||i + ||F(o)||i + 1) , 

where we use 3(^(0)) < C (||Q<z)(0)||i + ||F(0)||i + l) for the case Q>d(0) = and 3(^(0)) > 
mmin||<5>z)(0)||i for the other case Q>b(0) (i.e., ||Q>25(0)||i > 0). 
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